Ordinal Optimization: A Large Deviations Perspective

We consider the problem of optimal allocation of computing budget to maximize the probability of correct selection in the ordinal optimization setting. This problem has been studied in the literature in an approximate mathematical framework under the assumption that the underlying random variables from each population are independent and identically distributed with a Gaussian distribution. We use the large deviations theory to develop a mathematically rigorous framework for determining the optimal allocation of computing resources even when the underlying variables have general, non-Gaussian distributions. It follows from our analysis that in many realistic systems the assumption that batches of independent identically distributed random variables follow a Gaussian distribution may lead to significantly sub-optimal allocations. Further, in a simple setting we also show that when there exists an indifference zone, quick stopping rules may be developed that exploit the exponential decay rates of the probability of false selection.